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DETAILED ACTION 

1. This action is responsive to the amendment filed 1 1/23/05. 

2. Claims 1-14 and 16-21 are pending in the application. Claims 1,18, and 20 are 
independent claims. 

3. The rejection of claims 1, 2, 3, 7, 10, 1 1-17, and 18-20, under 35 U.S.C. 103(a) as being 
unpatentable over Hou in view of Telle have been withdrawn pursuant to applicant's 
amendment. The rejection of claims 4-6, 8, and 9, under 35 U.S.C. 103(a) as being unpatentable 
over Hou in view of Telle and further in view of Wolff have been withdrawn pursuant to 
applicant's amendment to the independent claims. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 1-4, 8-14, and 16-21, are rejected under 35 U.S.C 103(a) as being 
unpatentable over Skantze et al., USPN 6,722,574 filed (9/7/2001) in view of Ohmori et al., 
USPN 6,339,431 filed (9/14/1999). 

In reference to independent claim 1, Skantze teaches: 

The electronic circuitry part comprises image processing means for determining a 
position on the basis of the image recorded by the sensor and to carry out position determination 
on the basis of these images. A sheet of paper can be used to produce an electronic representation 
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of information which is being written or drawn on the surface (compare to "a memory for 
recording image data of a rendered hardcopy document'). The electronic representation can be 
produced by continuously determining, while writing on the surface with a pen. See column 8, 
lines 15-30 and column 3, lines 50-59. The processor and the hardcopy document taught by 
Skantze provide a method for recording image data of a rendered hardcopy document. 

The device can also comprise a display for showing positions or recorded information 
(compare to "a display screen"). See column 8, lines 30-35. 

When the function area is detected, the pen puts together a data packet with the following 
information: pen ED, function area, identity area, and a number of coordinates in the message 
field which correspond to the graph passed by the pen, a so-called pen graph (compare to "aw 
input device for recording signal data; the input device communicating the signal data to the 
processing device; the signal data representing at least a portion of the hardcopy document"). 
See column 11, lines 20-30. 

The position determination is carried out by a processor which thus must have software to 
locate and decode the symbols in an image and to determine positions on the basis of the thus 
obtained codes (compare to "a processing device coupled to the display screen"). See column 8, 
lines 45-56. 

The program passes through the pen graph and checks whether part of the pen graph, 
when subjected to intelligent character recognition, may be interpreted to be a sequence of digits 
corresponding to a fax number or a sequence of letters corresponding to an e-mail address, 
(compare to "identifying the image data in the signal data using the memory"). See column 12, 
lines 35-67. 
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The reference fails to explicitly disclose identifying annotations in the signal data by 
comparing the image data with the signal data and modifying the image data recorded in the 
memory in accordance with annotations. However, the Ohmori teaches an annotation 
transformation section that detects that the content of the dynamic image changes. Followed by a 
change the annotation transformation section eliminates the handwritten figure of the annotation 
in the dynamic image. See column 4, lines 50-67. The reference provides the suggestion of an 
annotation and modification provided to a user inputs. It would have been obvious to one of 
ordinary skill in the art, having the teachings of Skantze and Ohmori before him at the time the 
invention was made, to modify the marking methods taught by Skantze to include the annotation 
methods of Ohmori because it would have given the user a means of circling addresses on the 
business card to stress the importance of specific addresses. 

The business card may comprise check boxes indicating whether additional information 
is to be included in the e-mail. An example is a check box for private address particulars, i.e. in 
addition to company address and phone number (compare to 'memory records an identifier 
associated with the image data; the identifier being rendered on the hardcopy document; the 
identifier and the image data being recorded in the memory in response to rendering a 
hardcopy representation of the image data") See column 12, lines 45-67. 
In reference to dependent claim 2, Skantze teaches: 

The reference fails to explicitly disclose modifying the image data in accordance with the 
annotations, displaying the modified image, and updating the image data of the hardcopy 
document in the memory with the modified image data. However, the Ohmori teaches an 
annotation transformation section that detects that the content of the dynamic image changes. 
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Followed by a change the annotation transformation section eliminates the handwritten figure of 
the annotation in the dynamic image. See column 4, lines 50-67. The reference provides the 
suggestion of an annotation and modification provided to a user inputs. It would have been 
obvious to one of ordinary skill in the art, having the teachings of Skantze and Ohmori before 
him at the time the invention was made, to modify the marking methods taught by Skantze to 
include the annotation methods of Ohmori because it would have given the user a means of 
circling addresses on the business card to stress the importance of specific addresses. 
In reference to dependent claim 3, Skantze teaches: 

The reference provides image processing means for determining a position on the bases 
of the image recorded by the sensor and more specifically a processor unit with a processor 
which is programmed to read images. See column 8, lines 15-28. 
In reference to dependent claim 4, Skantze teaches: 

The reading pen is provided with a preferably optical sensor which reads the pattern of 
points and a processor which performs some image processing and calculation of the 
coordinates. See column 9, lines 30-37. 
In reference to dependent claim 8 and 9, Skantze teaches: 

An alternative to underlining is to encircle the address with a rectangle or some other 
closed figure, such as an ellipse. In certain circumstances, it is suitable for the processor of the 
pen to carry out the intelligent character recognition or at least parts thereof. See column 11, 
lines 60-67 and column 12, lines 1-5. 
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In reference to dependent claim 10, Skantze teaches: 

When the message is being written, the pen continually detects coordinates for the 
position of the pen by detecting with the sensor the pattern of points. See column 11, lines 9-15. 
In reference to dependent claim 11, Skantze teaches: 

The reading pen is provided with a preferably optical sensor which reads the pattern of 
points and a processor which performs some image processing and calculation of the 
coordinates. See column 9, lines 30-36. 
In reference to dependent claim 12, Skantze teaches: 

A program in the pen is activated, which packets a data of the pen, called pen ID, at least 
one coordinate from the send button area, and at least one coordinate from the identity area. See 
column 10, lines 5-17. 

In reference to dependent claim 13, Skantze teaches: 

The business card may comprise check boxes indicating whether additional information 
is to be included in the e-mail. An example is a check box for private address particulars, i.e. in 
addition to company address and phone number etc, also particulars about private address, phone 
number e-mail address etc. 

In reference to dependent claim 14, Skantze teaches: 

When the function area is detected, the pen puts together a data packet with the following 
information: pen ID, function area, identity area, and a number of coordinates in the message 
field which correspond to the graph passed by the pen. See column 11, lines 20-45. 
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In reference to dependent claim 16, Skantze teaches: 

The program passes through the pen graph and checks whether part of the pen graph, 
when subjected to intelligent character recognition, may be interpreted to be a sequence of digits 
corresponding to a fax number or a sequence of letters corresponding to an e-mail address. See 
column 11, lines 20-30. 

In reference to dependent claim 17, Skantze teaches: 

Instructions could be available that the fax number or the e-mail address should be 
underlined or crossed out for intelligent character recognition to be tried. See column 1 1, lines 
60-67. 

In reference to claims 18-21, the claims recite similar limitations used for performing the 
interactive document processing system as claimed in 1-4. In further view of the following, the 
claims are rejected under similar rationale. 

6. Claims 5 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Skantze in view of Ohmori and further view of Wolff UK Patent Application No. 2306669 
filed (10/30/1996). 

In reference to dependent claim 5, Skantze teaches: 

A reading pen is provided with a preferably optical sensor which reads the pattern of 
points and a processor which performs some image processing. See column 9, lines 30-35. The 
optics part comprises at least one light emitting diode for illuminating the surface, which is to be 
imaged, and a light-sensitive area sensor, such as a CCD or CMOS sensor, for recording a two- 
dimensional image. The device may also comprise a lens system. See column 8, lines 8-16. 
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However, the reference fails to disclose an embedded camera. Wolff discloses a stylus and a base 
unit for electronically processing and storing data received from the pen instrument, including 
processing and conditioning electronics and optical graphic screen display. See Wolff, page 17- 
21. It would have been obvious to one of ordinary skill in the art, having the teachings of Skantze 
and Wolff before him at the time the invention was made, to modify the image capturing device 
taught by Hou to include the stylus of Wolff, because it would have given the user the added 
benefit of having a stylus to read pre-printed bar code information and supporting a general 
manual entry interactive paper/electronic document handling system. 
In reference to dependent claim 6, Skantze teaches: 

A reading pen is provided with a preferably optical sensor, which reads the pattern of points and 
a processor which performs some image processing. See column 9, lines 30-35. The optics part 
comprises at least one light emitting diode for illuminating the surface, which is to be imaged, 
and a light-sensitive area sensor, such as a CCD or CMOS sensor, for recording a two- 
dimensional image. The device may also comprise a lens system. See column 8, lines 8-16. 
However, the reference fails to disclose an accelerometer for recording gestures. Wolff discloses 
a stylus and a base unit for electronically processing and storing data received from the pen- 
instrument, including processing and conditioning electronics with an optical graphic screen 
display. Wolff states that accelerometers may also be used for position sensors by double 
integration of the acceleration signals. See Wolff, page 18. It would have been obvious to one of 
ordinary skill in the art, having the teachings of Skantze and Wolff before him at the time the 
invention was made, to modify the image capturing device taught by Hou to include the 
accelerometer of Wolff, because it would have given the user the added benefit of having a 
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stylus to read pre-printed bar code information and supporting a general manual entry interactive 
paper/electronic document handling system. 

Response to Arguments 

7. Applicant's arguments with respect to claims have been considered but are moot in view 
of the new ground(s) of rejection. Applicant cancelled rejected claim 1 5 and included the 
limitation into the independent claim, which changes the scope of the claim when read as a 
whole. More specifically, the identifier recorded in memory and associated with the image data; 
the identifier being rendered on the hardcopy document; the identifier and the image data being 
recorded in the memory in response to rendering a hardcopy representation of the image data 
changes the scope of the invention. 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
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however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Matthew J. Ludwig whose telephone number is 571-272-4127. 
The examiner can normally be reached on 9:00am-6:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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